Image matching apparatus, image matching method, and image data output processing apparatus

ABSTRACT

In an image matching apparatus of the present invention, only a connected region in which the number of pixels included therein exceeds a threshold value, among connected regions that are specified by a labeling process section, is sent to a centroid calculation process section from a threshold value processing section, and a centroid (feature point) of the connected region is calculated. When it is determined that a target document to be matched is an N-up document, the threshold value processing section uses, instead of a default threshold value, a variant threshold value that varies depending on the number of images laid out on the N-up document and a document size that are found and detected by an N-up document determination section and a document size detection section. This makes it possible to determine a similarity to a reference document with high accuracy even in a case of an N-up document, i.e., a case where each target image to be matched is reduced in size from an original image.

This Nonprovisional application claims priority under U.S.C. §119(a) onPatent Application No. 234706/2007 filed in Japan on Sep. 10, 2007, theentire contents of which are hereby incorporated by reference.

FIELD OF THE INVENTION

The present invention relates to an image matching apparatus and animage matching method, each of which adopts an image matching techniquefor determining a similarity between an image of a target document to bematched and an image of a preliminarily stored reference document, andfurther relates to an image data output processing apparatus forcontrolling, in accordance with a similarity determination result, anoutput process such as a copying process, a transmitting process, anediting process, or a filing process with respect to input image data ofa target document.

BACKGROUND OF THE INVENTION

There have been proposed image matching techniques for comparing (i)image data obtained by reading a document by use of a scanner or thelike with (ii) image data of a preliminarily stored reference documentso as to determine a similarity between the image data and thepreliminarily stored image data.

Examples of the method for determining a similarity include: a method inwhich a keyword is extracted from an image with OCR (Optical CharacterReader) so as to carry out matching with the keyword; a method in whichonly a ruled line image having a ruled line is focused on as a targetimage, and matching is carried out with features of the ruled line (seePatent Document 1); and a method in which a similarity is determinedbased on color distributions of an input image and a storage image (seePatent Document 2).

Patent Document 3 discloses a technique in which a descriptor is formedfrom features of an input document, and matching between the inputdocument and a document stored in a document database is carried out byuse of the descriptor and a descriptor database in which descriptors arestored and which indicates a list of documents including features fromwhich the descriptors are formed. A descriptor is selected such that thedescriptor is invariant for distortions generated by digitalization of adocument and differences between an input document and a document usedfor matching in a document database.

In this technique, when the descriptor database is scanned, votes foreach document in the document database are accumulated, and a documenthaving the maximum number of votes obtained or a document whose numberof votes exceeds a threshold value is used as a matched document.

Further, Patent Document 4 discloses a technique in which a plurality offeature points are extracted from a digital image, a set of localfeature points are determined out of the extracted feature points, apartial set of feature points is selected out of the determined set oflocal feature points, invariants relative to geometric transformationeach as a value characterizing the selected partial set are calculatedin accordance with plural combinations of feature points in the partialset, features are calculated from combinations of each of the calculatedinvariants, and a document and an image corresponding to the digitalimage data is searched by voting documents and images having thecalculated features stored in a database.

Conventionally, in an image data output processing apparatus, e.g., acopying machine, a facsimile device, a scanning device, or amulti-function printer, which carries out, with respect to input imagedata (image data of a target document to be matched), an output processsuch as a copying process, a transmitting process, an editing process,or a filing process, when it is determined that an input image of atarget document is similar to an image of a reference document by use ofsuch the image matching techniques, its output process is controlled.

For example, there has been known techniques of a color image formingapparatus as anti-counterfeit techniques with respect to a papercurrency or a valuable stock certificate, in which it is determinedwhether or not input image data is identical with an image of a papercurrency or a valuable stock certificate in accordance with a patterndetected from the input image data, and when it is determined that theinput image data is identical with a reference image, (i) a specifiedpattern is added to an output image so that an image forming apparatusthat has made a copy of the image data can be specified from the outputimage, (ii) a copied image is blacked out, or (iii) a copying operationis prohibited with respect to the input image data.

Patent Document 1: Japanese Unexamined Patent Publication, Tokukaihei,No. 8-255236 (published on Oct. 1, 1996)

Patent Document 2: Japanese Unexamined Patent Publication, Tokukaihei,No. 5-110815 (published on Apr. 30, 1993)

Patent Document 3: Japanese Unexamined Patent Publication, Tokukaihei,No. 7-282088 (published on Oct. 27, 1995)

Patent Document 4: International Publication No. WO 2006/092957,pamphlet (published on Sep. 8, 2006)

However, such a conventional image matching apparatus has a problem inwhich, in a case where a target document is an N-up document or areduced-size document, it is difficult to precisely determine asimilarity to a reference document.

Conventionally, there have been known some image matching apparatuses inwhich input image data is binarized, a connected region in which pixelsin the binarized image are connected to each other is specified, afeature point of the connected region thus specified is extracted basedon coordinates, in the binarized image, of each pixel included in theconnected region, features indicative of a similarity of the image arecalculated based on the extracted feature point, and similaritydetermination is carried out.

In such the similarity determination, a difference in a condition forspecifying a connected region decreases accuracy in determination. Thedifference in a condition for specifying a connected region means astate in which a partial image that is specified as a connected regionin one of two images for comparison is not specified as a connectedregion in another one of the two images.

As the connected region, a region that includes more than a specifiednumber of pixels (the default number of pixels) is specified. This isbecause an isolated dot, a noise, and the like are to be removed.

However, in the conventional image matching apparatuses, a thresholdvalue for removing such an isolated dot and a noise is fixed to adefault threshold value. The default threshold value is a value capableof removing an isolated dot and a noise in image data of a referencedocument. In a case where a connected region is specified in an imagereduced in size from an original image size (e.g., an N-up document, areduced-size document, and the like) by use of the same threshold valueas the reference document, in a part in which the number of pixelsactually exceeds the threshold value and which should be specified as aconnected region, the number of pixels cannot exceed the threshold valuebecause the image is reduced in size, thereby resulting in that the partis not specified as the connected region. Consequently, the number offeature points decreases and features differ, thereby decreasingaccuracy in determination.

SUMMARY OF THE INVENTION

An object of the present invention is to provide an image matchingapparatus, an image matching method, and an image data output apparatusincluding the same, in each of which it is determined whether or not animage of a target document to be matched is similar to an image of areference document, and each of which can determine a similarity to areference document with high accuracy even in a case of an N-updocument, i.e., a case where each target image to be matched is reducedin size from an original image.

In order to achieve the above object, an image matching apparatus of thepresent invention includes a features extraction section for extractingfeatures of an image from input image data, the features extractionsection extracting the features based on a connected region in whichpixels in a binarized image of the image data are connected to eachother and the number of the pixels exceeds a default threshold value,said apparatus (i) causing the features extraction section to extractthe features from input image data of a target document to be matched,and (ii) determining whether or not the target document has a similarimage to that of a preliminarily stored reference document based on acomparison of the features thus extracted with features of the image ofthe reference document. The image matching apparatus further includes anN-up document determination section that determines (i) whether or notthe target document is an N-up document on which plural documents arelaid out, and (ii) the number of images laid out on the N-up documentwhen it is determined that the target document is an N-up document; anda document size detection section that detects a document size of thetarget document, when the N-up document determination section determinesthat the target document is an N-up document, the features extractionsection extracting the features, by use of not the default thresholdvalue but a variant threshold value that varies depending on the numberof the images laid out on the N-up document and the document size thatare found and detected by the N-up document determination section andthe document size detection section, respectively, based on a connectedregion in which the number of pixels exceeds the variant thresholdvalue.

Image data that is inputted into the image matching apparatus includes:image data that is scanned and inputted by a scanner; and further,electronic data formed by use of a computer (software), e.g., electronicdata that is formed by filling in a form of the electronic data withnecessary information by use of a computer (software).

In the arrangement, in the case where the N-up document determinationsection determines that a target document to be matched is an N-updocument, the features extraction section extracts the features by useof not the default threshold value but a variant threshold value thatvaries depending on the number of the images laid out on the N-updocument and the document size that are found and detected by the N-updocument determination section and the document size detection section,respectively, based on a connected region in which the number of pixelsexceeds the variant threshold value.

With the arrangement, even when the target document is an N-up documentand images are reduced in size, a threshold value that is used forextracting of a connected region to be subjected to features calculationis the variant threshold value that varies depending on the documentsize and the number of the images laid out on the N-up document, thatis, a threshold value that varies depending on a reduction ratio of thetarget document, thereby improving accuracy in matching in an N-updocument.

In order to achieve the above object, an image matching method of thepresent invention includes the step of: (a) extracting features of animage from input image data based on a connected region in which pixelsin a binarized image of the image data are connected to each other andthe number of the pixels exceeds a default threshold value, wherein thefeatures are extracted from input image data of a target document to bematched in the step (a), and it is determined whether or not the targetdocument has a similar image to that of a preliminarily stored referencedocument, based on a comparison of the features thus extracted withfeatures of the image of the reference document. The method, furtherincludes the steps of: (b) determining (i) whether or not the targetdocument is an N-up document on which plural documents are laid out, and(ii) the number of images laid out on the N-up document when it isdetermined that the target document is an N-up document; and (c)detecting a document size of the target document, when it is determinedthat the target document is an N-up document in the step (b), the step(a) is carried out such that the features are extracted, by use of notthe default threshold value but a variant threshold value that variesdepending on the number of the images laid out on the N-up document andthe document size that are found and detected by the N-up documentdetermination section and the document size detection section,respectively, based on a connected region in which the number of pixelsexceeds the variant threshold value.

In the similar manner to the image matching apparatus of the presentinvention that has been already described, in the image matching method,it is possible to extract features under the same condition as thereference document even when the target document is an N-up document andimages are reduced in size from their original sizes. This makes itpossible to accurately determine a similarity to a reference document.

Further, the present invention also includes an image data outputprocessing apparatus that carries out an output process with respect toimage data, the apparatus including: the image matching apparatus of thepresent invention; and an output process control section that controlsan output process with respect to image data of the target document inaccordance with a determination result of the image matching apparatus.

As described above, the image matching apparatus of the presentinvention can determine a similarity with high accuracy, even in a casewhere the target document is an N-up document and images thereof arereduced in size from their original size.

Accordingly, in a case where image data to be subjected to the outputprocess is similar to that of a reference document, the image dataoutput processing apparatus including such the image matching apparatuscan control the output process with high accuracy. This can increasereliability.

Furthermore, the image matching apparatuses may be realized by acomputer. In this case, the present invention includes acomputer-readable storage medium in which an image processing programthat causes a computer to function as respective sections in the imagematching apparatus so that the image matching apparatus is realized inthe computer is stored.

Additional objects, features, and strengths of the present inventionwill be made clear by the description below. Further, the advantages ofthe present invention will be evident from the following explanation inreference to the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram schematically illustrating an arrangement of acentroid calculation section of a document matching process sectionprovided in an image data output processing apparatus as a digital colorcopying machine according to one embodiment of the present invention.

FIG. 2 is a block diagram schematically illustrating an image dataoutput processing apparatus as a digital color copying machine accordingto one embodiment of the present invention.

FIG. 3 is an explanatory drawing illustrating a process of determining(i) whether or not a target document to be matched is an N-up documentand (ii) the number of images laid out on the N-up document in adocument type discrimination section provided in the image data outputprocessing apparatus in FIG. 2.

FIG. 4 is an explanatory drawing illustrating a process of determining(i) whether or not a target document to be matched is an N-up documentand (ii) the number of images laid out on the N-up document in thedocument type discrimination section provided in the image data outputprocessing apparatus in FIG. 2.

FIG. 5 is a block diagram schematically illustrating an arrangement of adocument matching process section provided in the image data outputprocessing apparatus in FIG. 2.

FIG. 6 is a block diagram schematically illustrating an arrangement of afeature point calculation section provided in the image data outputprocessing apparatus in FIG. 2.

FIG. 7 is a block diagram schematically illustrating an arrangement of afeatures calculation section provided in the image data outputprocessing apparatus in FIG. 2.

FIG. 8 is an explanatory drawing illustrating an example of filtercoefficients for a mixing filter provided in an MTF process section ofthe image data output processing apparatus in FIG. 2.

FIG. 9 is an explanatory drawing illustrating an example of a connectedregion extracted from input image data by the feature point calculationsection of the image data output processing apparatus in FIG. 2, and acentroid thereof.

FIG. 10 is an explanatory drawing illustrating an example of eachcentroid (feature point) of a plurality of connected regions extractedfrom a text string included in input image data by the feature pointcalculation section of the image data output processing apparatus inFIG. 2.

FIG. 11 is an explanatory drawing illustrating an example of a targetfeature point and peripheral feature points that are extracted at a timewhen features are calculated by the features calculation section of theimage data output processing apparatus in FIG. 2.

FIG. 12 (a) through FIG. 12 (d) are explanatory drawings eachillustrating an example of a combination of a target feature point andperipheral feature points extracted at a time when the featurescalculation section of the image data output processing apparatus inFIG. 2 calculates features.

FIG. 13 (a) through FIG. 13 (d) are explanatory drawings eachillustrating a combination of a target feature point and peripheralfeature points extracted at a time when the features calculation sectionof the image data output processing apparatus in FIG. 2 calculatesfeatures.

FIG. 14 (a) and FIG. 14 (b) are explanatory drawings each illustratingan example of a hash value of each feature point stored in a hash tableand an index indicative of input image data.

FIG. 15 is a graph showing an example of the number of votes for eachreference image in a voting process section of the image data outputprocessing apparatus in FIG. 2.

FIG. 16 (a) is an explanatory view illustrating a labeling process in alabeling process section of a centroid calculation section provided inthe image data output processing apparatus illustrated in FIG. 2, andFIG. 16 (b) is an explanatory view illustrating a process of addingcoordinate values of target pixels to be labeled to a buffer foraccumulating coordinate values, in regard to each label, in the labelingprocess section.

FIG. 17 is an explanatory view illustrating four pixels that areadjacent to a current pixel X and determines which label is assigned tothe current pixel X, in the labeling process in the labeling processsection of the centroid calculation section provided in the image dataoutput processing apparatus illustrated in FIG. 2.

FIG. 18 is an explanatory view illustrating a merging process that iscarried out when different labels are assigned to the four pixels, whichare adjacent to the current pixel X and determines which label isassigned to the current pixel X in the labeling process in the labelingprocess section of the centroid calculation process provided in theimage data output processing apparatus illustrated in FIG. 2.

FIG. 19 is an explanatory view illustrating a multiplier coefficient fora default threshold value, the multiplier coefficient being set so as tovary depending on a document size of a target document to be matched andthe number of images laid out on the N-up document, in a thresholdprocess section of the centroid calculation section provided in theimage data output processing apparatus illustrated in FIG. 2.

FIG. 20 is a flowchart illustrating a flow of a similarity determinationprocess in the image data output processing apparatus in FIG. 2.

FIG. 21 is a block diagram schematically illustrating an arrangement ofan image data output processing apparatus as a digital colormulti-function printer according to another embodiment of the presentinvention.

FIG. 22 is an explanatory drawing illustrating a flow of image data in afacsimile process in the image data output processing apparatus in FIG.21.

FIG. 23 is a block diagram schematically illustrating an arrangement ofan image data output processing apparatus according to further anotherembodiment of the present invention.

DESCRIPTION OF THE EMBODIMENTS

An embodiment of the present invention is explained below. Explained inthe embodiment is a case where the present invention is applied to adigital color copying machine, however, the present invention is notlimited to this.

FIG. 2 is a block diagram schematically illustrating a digital colorcopying machine (image data output processing apparatus) 1 according tothe present embodiment.

As illustrated in FIG. 2, the digital color copying machine 1 includes acolor image input apparatus 2, a color image processing apparatus 3, acolor image output apparatus 4, and an operation panel 6.

The color image input apparatus 2 that reads out an image of a documentand generates image data, is, for example, composed of a scanner (notshown) including a CCD (Charge Coupled Device) for converting opticalinformation into an electric signal. The color image input apparatus 2reads an optical image reflected from a document as RGB (R: Red, G:Green, B: Blue) analog signals and supplies the RGB analog signals tothe color image processing apparatus 3.

The color image processing apparatus 3 carries out various processeswith respect to the analog signals received from the color image inputapparatus 2 so that the analog signals are converted into a format thatcan be handled by the color image output apparatus 4, and supplies thesignals thus processed to the color image output apparatus.

The color image processing apparatus 3 includes, at an input stage, anA/D (Analog/Digital) conversion section 11 for converting RGB analogsignals into digital signals. The A/D conversion section 11 converts theimage data in the form of the analog signals received by the color imageprocessing apparatus 3 into digital signals.

The RGB signals converted into the digital signals are then transmittedto a shading correction section 12, a document type discriminationsection 13, a document matching process section 14, an input tonecorrection section 15, an editing process section 16, a segmentationprocess section 17, a color correction section 18, a black generationand under color removal section 19, a spatial filter process section 20,an output tone correction section 21, and a tone reproduction processsection 22 in this order. Ultimately, the RGB signals are converted intoCMYK signals as digital signals. The CMYK digital signals outputted fromthe tone reproduction process section 22 are temporarily stored in amemory (not shown) and then supplied to the color image output apparatus4.

The following explanation deals with each process of the sectionsconstituting the color image processing apparatus 3. The shadingcorrection section 12 removes various distortions produced in anillumination system, an image focusing system, and an image sensingsystem of the color image input apparatus 2 from the RGB digital signalstransmitted from the A/D converter 11. Moreover, the shading correctionsection 12 adjusts color balance of the RGB digital signals and convertseach signal into a signal such as a density (pixel value) signal whichcan be processed easily by an image processing system used in the colorimage processing apparatus 3.

The RGB signals (RGB density (pixel value) signals) whose variousdistortions have been removed and color balance has been adjusted by theshading correction section 12 are then supplied to the document typediscrimination section 13.

The document type discrimination section 13 discriminates a documenttype of the input image data in accordance with the RGB signalstransmitted from the shading correction section 12. Namely, the documenttype discrimination section 13 determines whether the document is a textdocument, a photographic-picture, which means a continuous tone image,for example, silver halide photography, a text/photographic-picturedocument that includes a text and a photographic-picture, or the likedocument. A document type discrimination result is used in subsequentprocesses.

In this embodiment, the document type discrimination section (N-updocument determination section) 13 also determines whether the documentfrom which the input image data is read out is an N-up document or not.If the document is determined as an N-up document, the document typediscrimination section 13 further calculates the number of images laidout on the N-up document.

The N-up document means that a single-sheet document includes aplurality of document images laid out thereon, and is, for example, anN-in-1 (N=2, 4, 6, 8, 9 . . . ) document, i.e., a single-sheet documenton which N sheets of document images are laid out collectively.Processes, in the document type discrimination section 13, ofdiscriminating an N-up document and calculating the number of imageslaid out on the N-up document will be described later.

Meanwhile, the document type discrimination section 13 transmits theinput signals as received from the shading correction section 12,without any modification, to the subsequent document matching processsection 14.

The document matching process section (similarity determination section)14 has a function of extracting features of an image of the image datafrom the RGB signals (input image data) transmitted from the documenttype discrimination section 13.

While extracting features of an image of input image data (a targetimage to be matched), the document matching process section 14 furthercarries out a similarity determination process of determining whether ornot the image of the input image data is similar to that of apreliminarily stored reference document (hereinafter also referred to asa reference image) whose features have been extracted by the function ofextracting features, based on a comparison of the features thusextracted with features of the image of the reference document.

When it is determined that the image of the input image data is similarto the reference image in the similarity determination process, thedocument matching process section 14 then supplies a control signal tocontrol an output process such as prohibiting an output process withrespect to the input image data (an image forming process in the case ofthe color copying machine), degrading an image quality, or filing of theimage data. More details of the document matching process section 14will be described later.

Meanwhile, the document matching process section 14 transmits the inputsignals as received from the document type discrimination section 13,without any modification, to the input tone correction section 15.

The input tone correction section 15 removes background color (densitycomponent of the background: background density) from the RGB signalstransmitted from the document matching process section 14, and adjustsimage quality such as contrast.

In a case where the document is an N-up document and requires to beedited, the editing process section 16 edits the RGB signals transmittedfrom the input tone correction section 15, from which signals backgroundcolor (density component of the background: background density) has beenremoved and for which image quality such as contrast has been adjusted.

For example, in a case where an N-up document is to be outputted suchthat plural documents laid out on the N-up document are separatelyoutputted one by one, the editing process section 16 determinespositions to divide the N-up document, divides an image (of the N-updocument) according to the positions, rotates and enlarges each ofdivided images, and edits each image so that the image is converted intoseparated image data.

In a case where the RGB signals transmitted from the input tonecorrection section 15 are not the ones of an N-up document, or the RGBsignals are the ones of an N-up document but do not require the editingprocess, the RGB signals as received from the input tone correctionsection 15 are transmitted, without any modification, to thesegmentation process section 17 from the editing process section 16.

The RGB signals edited by the editing process section or the RGB signalstransmitted, without any modification, from the editing process section16 are sent to the segmentation process section 17.

The segmentation process section 17 separates each pixel of an inputimage into either one of a text region, a halftone dot region, or aphotograph (continuous tone) region, according to the RGB signals. Onthe basis of a result of the separation, the segmentation processsection outputs a segmentation class signal, indicating which region apixel of the input image belongs to, to the color correction section 18,the black generation and under color removal section 19, the spatialfilter process section 20, and the tone reproduction process section 22.The segmentation process section 17 also outputs the input signals asreceived from the input tone correction section to the subsequent colorcorrection section 18 without any modification.

The color correction section 18 removes color impurity on the basis ofspectral characteristics of CMY (C: Cyan, M: Magenta, and Y: Yellow)color materials including a useless absorption component, in order torealize a faithful color reproduction.

The black generation and under color removal section 19 performs (i) ablack generation process for generating a black (K) signal from threecolor (CMY) signals after the color correction process and (ii) ageneration process of new CMY signals by removing the K signal obtainedby the black generation process from the original CMY signals. As aresult, the three CMY signals are converted into four CMYK signals.

With the use of a digital filter, the spatial filter process section 20performs a spatial filter process on the basis of a segmentation classsignal, with respect to the image data which is received in the form ofthe CMYK signals from the black generation and under color removalsection 19. In the spatial filter process, the spatial filter processsection 20 corrects a spatial frequency characteristic, so as to reduceblur or granularity deterioration in an output image. The tonereproduction process section 22, as with the spatial filter processsection 20, performs a predetermined process with respect to the imagedata in the form of the CMYK signals, on the basis of the segmentationclass signal.

For example, in the region separated into a text region by thesegmentation process section 17, the spatial filter process section 20strongly emphasizes a high frequency component in an edge enhancementprocess of the spatial filter process, in order to improvereproducibility of a black text or a color text especially.Concurrently, the tone reproduction process section 22 selects either abinarization process or a multi-level dithering process on ahigh-resolution screen suitable for reproducing the high frequencycomponent.

In the region separated into a halftone dot region by the segmentationprocess section 17, the spatial filter process section 20 performs a lowpass filter process for removing an input halftone dot component. Theoutput tone correction section 21 performs an output tone correctionprocess in which a signal such as a density (pixel value) signal isconverted into a halftone dot area rate that is characteristics of thecolor image output apparatus 4. Subsequently, the tone reproductionprocess section 22 performs a tone reproduction process (half tonegeneration) so that, ultimately, an image is segmented into pixels andeach tone of the pixels can be reproduced. In the region separated intoa photograph region by the segmentation process section 17, thebinarization process or the multi-level dithering process is performedon a screen suitable for tone reproduction.

Furthermore, in the present embodiment, the document type discriminationresult is taken into consideration in each process performedsubsequently to the segmentation process. In each process, in a casewhere it is determined that the input image does not include pluralregions collectively, the segmentation process is performed in a similarmanner to the above. On the other hand, in a case where it is determinedthat the input image includes plural regions, the segmentation processis performed with the use of an intermediate parameter betweenparameters used in each region process. In this case, a parameter for aregion that has not been discriminated in the document typediscrimination process is not used.

For example, when an input image is determined as a text document, thesegmentation process is performed in such a manner that a regionseparated into a text and a line drawing is considered effective, whilea region, such as a halftone dot and a photograph, which is separatedinto a continuous tone, is regarded as an erroneous separation and theregion is not reflected to the segmentation process. This is because,even if the input image is a text document, the region determined as acontinuous tone may be discriminated wrongly depending on a documenttype.

On the basis of a result of the segmentation process, the input tonecorrection process and the tone reproduction process remove morehighlight, and use a correction curve for making a high contrast.

While a color correction process focusing on a color saturation (chroma)is carried out with respect to a color text, a generous amount of blackis set with respect to a black text in the black generation and undercolor removal process. With respect to a text, the spatial filterprocess emphasizes an edge of the text, and a parameter is switchedover, e.g., setting of filter coefficients is performed so that asmoothing process is mildly carried out.

When an input image is determined as a text/photographic-picturedocument, each process is carried out by use of an intermediateparameter between parameters used in a text document process and aphotographic-picture document process. In the segmentation process, aregion separated into a text, a line drawing, or a photograph isconsidered effective, while a region separated into a halftone dot isregarded as an erroneous separation and the region is not reflected tothe segmentation process. This is because, even if the input image is atext/photographic-picture document, the region separated into a halftonedot may be discriminated wrongly depending on a document type.

Depending on which of the text document and the photographic-picturedocument is focused on, in the input tone correction process and thetone reproduction process, highlight is removed and contrast is adjustedby use of a parameter intermediate between parameters used in thephotographic-picture document process and the text document process, andthe color correction process is performed so that a color saturation(chroma) strength and a tone balance are not immoderate. Meanwhile, inthe black generation and under color removal process, an amount of blackis adjusted moderately so as not to affect the photographic-pictureimage.

The input image data that has been subjected to each of the processesmentioned above is temporarily stored in a memory (not shown). The inputimage data is read out from the memory at a predetermined timing andsupplied to the color image output apparatus 4. Then, the input imagedata is subjected to the similarity determination process by thedocument matching process section 14. In a case where there is asimilarity between the image of the input image data and a referenceimage, a control signal to prohibit the output process with respect tothe input image data is supplied. In this case, when the image data isread out from the memory, a process of deleting or blacking out theimage data is carried out so that the target image does not appear,thereby resulting in that the input image data is outputted as a blankpaper or a blacked-out image.

The color image output apparatus 4 outputs the image data supplied fromthe color image processing apparatus 3 to a recording medium such aspaper. The color image output apparatus 4 is not particularly limitedand may be a color image output apparatus which uses anelectrophotographic method or an ink-jet method.

The operation panel 6 includes, for example, a display section (notshown) such as a liquid crystal display and setting buttons (not shown).The operation panel 6 causes the display section to display informationcorresponding to an instruction of a main control section (not shown) ofthe digital color copying machine 1 and transmits information, inputtedby the user with use of the setting buttons, to the main controlsection. The user can input, from the operation panel 6, a processingrequest, the number of images to be processed, or the like with respectto the image data.

The main control section is composed of a CPU (Central Processing Unit)and the like for example. In accordance with a program or various datastored in a ROM (not shown) or the like and information which is enteredfrom the operation panel 6, the main control section controls eachoperation of each section of the digital color copying machine 1.

The following explanation deals with processes, in the document typediscrimination section 13, of determining whether a target document isan N-up document or not, and of detecting the number of images laid outon the N-up document.

The document type discrimination section 13 calculates, from RGB signals(image data), distributions of the number of density transition (or thenumber of edges), in which a pixel value varies from 0 to 1 or viceversa, in each line of an image in main and sub scanning directions. Onthe basis of the distributions, the document type discrimination section13 determines whether a target document is an N-up document or not, anddetects the number of images laid out on the N-up document when thetarget document is determined as an N-up document.

FIG. 3 illustrates a histogram of the number of density transition in anon N-up document in each of main and sub scanning directions, which nonN-up document includes an image(s) in which (i) a text string extends inthe sub scanning direction that is a paper-lateral direction(document-lateral direction) and (ii) line spaces are arranged side byside in the main scanning direction that is a paper-longitudinaldirection (document-longitudinal direction).

In such the non N-up document, as illustrated in FIG. 3, thedistribution of the number of density transition in the main scanningdirection includes the number of density transition at everypredetermined space corresponding to a space between text lines. On theother hand, the distribution of the number of density transition in thesub scanning direction is continuous except for a blank space in aperiphery of the document.

An N-up document on which two documents, such as the one illustrated inFIG. 3, are printed in a 2-in-1 mode is arranged such that, asillustrated in FIG. 4, two images are laid out side by side in a mainscanning direction (a paper-longitudinal direction of the N-updocument). In each of the images, a text string extends in the mainscanning direction (a paper-lateral direction in their originaldocuments) and line spaces are arranged side by side in a sub scanningdirection (a paper-longitudinal direction in the original documents).

In such the N-up document, as illustrated in FIG. 4, in a histogram ofthe number of density transition in each of the main and sub scanningdirections, there are two continuous distributions of the number ofdensity transition in the main scanning direction. A blank regioncorresponding to blank spaces of the original documents exists betweenthese distributions of the number of density transition. On the otherhand, a distribution of the number of density transition in the subscanning direction includes the number of density transition at everypredetermined spaces corresponding to line spaces.

On the basis of the histogram, in a case where (i) there are twocontinuous distributions of the number of density transition in the mainscanning direction, and (ii) there is a blank region between the twocontinuous distributions, whose number of density transition is not morethan a predetermined value (e.g., 20) and the blank region is around 20mm in width (i.e., around 170 lines in a case where a resolution is 300dpi) (although it depends on a layout of an N-up document), it isdetermined that a target image to be matched is that of a 2-in-1document.

Note, however, that, in a case where it is determined whether a targetdocument includes a blank region or not, when a first line or a lastline is included in a line whose number of density transition is notmore than a predetermined value, the line is determined as a blank spacein a periphery of the target document, and requires to be excluded.

As such, it is determined whether or not a target document includes ablank region, by calculating the distributions of the number of densitytransition in the main and sub scanning directions. This makes itpossible to determine whether the target document is a 2-in-1 documentor not.

The above explanation deals with how to discriminate a 2-in-1 documentwhose number of images laid out thereon is 2. In a case of a 4-in-1document whose number of images laid out thereon is 4, the document hasblank regions corresponding to blank spaces to separate each documentimage in the main and sub scanning lines. Accordingly, it is possible todetermine whether a target document is a 4-in-1 document or not in thesimilar manner.

In this embodiment, the distributions of the number of densitytransition (or the number of edges) in each line are calculatedrespectively in the main and sub scanning directions. Alternatively, anaverage value or a variance value of a pixel value per line may be alsoused.

As a method other than the method of discriminating an N-up document byuse of such image data, it is also possible to discriminate an N-updocument by setting conditions.

For example, in a case where a split-image output mode is selected as animage mode by input operation with the use of the operation panel 6, amain control section (CPU) for controlling operations of sections in thedigital copying machine 1 recognizes this mode. Accordingly, it can bedetermined that input image data is that of an N-up document.

Alternatively, in a case where the color image input apparatus 2 is ascanner that is connected to a computer, a document type is selected ina setting screen for readout conditions (a setting screen of a scannerdriver), so that the main control section (CPU) recognizes the selectionresult. Herewith, an N-up document can be discriminated.

Next explained is the document matching process section (a documentmatching process apparatus) 14 in detail. The document matching processsection 14 according to the present embodiment extracts plural featurepoints from the input image data, and determines a set of local featurepoints relative to each of the extracted feature points. Then, thedocument matching process section 14 selects a partial set of featurepoints out of the determined set of local feature points, and calculatesinvariants each of which is relative to geometric transformation as avalue characterizing the selected partial set in accordance with pluralcombinations of feature points in the partial set. The section 14calculates a hash value (features (feature vectors)) by combining thecalculated invariants, and votes for a reference image corresponding tothe hash value. Thereby, a reference image similar to the input imagedata is retrieved and a similarity determination process is carried outwith respect to the reference image.

FIG. 5 is a block diagram schematically illustrating an arrangement ofthe document matching process section 14. As illustrated in FIG. 5, thedocument matching process section 14 includes a feature pointcalculation section 31, a features calculation section 32, a votingprocess section 33, a similarity determination process section 34, astorage process section 35, a control section 7, and a memory 8. Afeatures extraction section of the present invention is constituted bythe feature point calculation section 31 and the features calculationsection 32.

The control section 7 controls operations of the sections of thedocument matching process section 14. Note that, the control section 7may be provided in the main control section for controlling operationsof the sections of the digital color copying machine 1 or may beprovided separately from the main control section so as to cooperatewith the main control section in controlling operations of the documentmatching process section 14.

When it is determined that there is no similarity (an image of the inputimage data is not identical with the reference image) in accordance witha result of the similarity determination process by the documentmatching process section 14, the control section (output process controlsection) 7 supplies a control signal to permit an output process withrespect to the image data. On the other hand, when it is determined thatthere is a similarity (an image of the input image data is identicalwith the reference image), the control section 7 supplies a controlsignal to control an output process with respect to the image data(hereinafter referred to as the input image data).

The memory 8 stores various data to be used in processes of the sectionsof the document matching process section 14, process results, and thelike.

The feature point calculation section 31 extracts a connected part of atext string or a line from the input image data and calculates acentroid of the connected part as a feature point. Here, in a storageprocess of a reference image, the input image data indicates image dataof an image to be stored. On the other hand, in the similaritydetermination process, the input image data indicates image data of atarget image to be matched (hereinafter also referred to as target imagedata in some cases).

FIG. 6 is a block diagram schematically illustrating an arrangement ofthe feature point calculation section 31. The arrangement illustrated inFIG. 6 is one exemplary arrangement of the feature point calculationsection 31, and not limited to this. For example, a feature point may becalculated by various conventional methods.

As illustrated in FIG. 6, the feature point calculation section 31includes a signal conversion process section 41, a resolution conversionsection 42, an MTF process section 43, a binarization process section44, and a centroid calculation section 45.

In a case where image data (RGB signals) inputted from the shadingcorrection section 12 is a color image, the signal conversion processsection 41 achromatizes the image data and converts the achromatizedimage data into a lightness signal or a luminance signal.

For example, the signal conversion process section 41 converts the RGBsignals into a luminance signal Y in accordance with the followingexpression (1).Yj=0.30Rj+0.59Gj+0.11Bj  (1)

“Yj” refers to a luminance signal of each pixel, and each of Rj, Gj, andBj is a color component of the RGB signals, and “j” subsequently addedto “Y”, “R”, “G”, and “B” represents a value given to each pixel (j isan integer not less than 1).

Alternatively, the RGB signals may be converted into CIE1976L*a*b*signal (CIE: Commission International de l' Eclairage, L*: lightness,a*, b*: chromaticity).

The resolution conversion section 42 scales up/down the input imagedata. For example, in a case where the input image data is opticallyscaled up/down by the image input apparatus 2, the resolution conversionsection 42 scales up/down the input image data again so as to have apredetermined resolution (hereinafter referred to as a defaultresolution).

Further, in order to reduce processes carried out by the subsequentprocess sections, the resolution conversion section 42 may convertresolution so as to make the resolution lower than a resolution in beingscanned by the image input apparatus 2 at an equal scale factor (forexample, image data scanned at 600 dpi (dot per inch) is converted intodata of 300 dpi or a similar operation is carried out).

The MTF (modulation transfer function) process section 43 is used tocover (adjust) unevenness of spatial frequency characteristics among aplurality of color image input apparatuses 2. In the image signaloutputted from a CCD, MTF deterioration occurs due to an opticalcomponent such as a lens or a mirror, an aperture in a light receivingsurface of the CCD, transfer efficiency or afterimage, storage effect orscanning unevenness caused by physical scanning, and a similar cause.The MTF deterioration causes the scanned image to blur.

The MTF process section 43 carries out an appropriate filtering process(emphasizing process) so as to recover the blur caused by the MTFdeterioration. Further, the MTF process section 43 is used also tosuppress a high frequency component that is unnecessary in alater-mentioned feature point extracting process in the subsequentcentroid calculation section 45. That is, a mixing filter (not shown) isused to carry out an emphasizing process and a smoothing process. Notethat, FIG. 8 illustrates an example of a filter coefficient of themixing filter.

The binarization process section 44 compares the achromatized image data(a luminance value (luminance signal) or a lightness value (lightnesssignal)) with a predetermined threshold so as to binarize the imagedata.

The centroid calculation section 45 carries out labeling with respect toeach pixel (labeling process) in accordance with the image databinarized by the binarization process section 44 (for example, the imagedata is represented by “1” and “0”). The centroid calculation section 45specifies a connected region in which pixels each having the same labelare connected to each other, extracts a centroid of the specifiedconnected region as a feature point, and further, sends the featurepoint to the features calculation section 32. The feature point can berepresented by coordinate value (x-coordinate and y-coordinate) in thebinarized image.

The centroid calculation section 45 calculates a feature point(centroid) from a connected region in which the number of pixels exceedsa default threshold value. Meanwhile, in a case where the number of thepixels included in the connected region is less than the defaultthreshold value, the centroid calculation section 45 does not specifythe connected region as the one to be subjected to the centroidcalculation, and does not carry out the centroid calculation withrespect to the connected region. This avoids extracting a feature pointfrom an isolated dot and a noise. On this account, the default thresholdvalue is set to a value (the number of pixels) that is capable ofadequately removing an isolated dot and a noise, based on a performanceof an image readout apparatus to be used, an image sample to be readout, and the like.

In the present embodiment, in a case where a target document to bematched is not an N-up document, the centroid calculation section 45determines, by use of the default threshold value, whether or not aconnected region is the one to be subjected to the centroid calculation.On the other hand, in a case where the target document is an N-updocument, the centroid calculation section 45 determines whether or nota connected region is the one to be subjected to the centroidcalculation, by use of a variant threshold value which varies dependingon the number of images laid out on the N-up document and a documentsize of input image data. More details will be described afterexplanation of a similarity determination process.

FIG. 9 is an explanatory diagram illustrating an example of a connectedregion extracted from the input image data and a centroid of theconnected region. FIG. 9 illustrates a connected region corresponding toa text string “A” and its centroid. FIG. 10 is an explanatory viewillustrating an example of centroids (feature points) of pluralconnected regions extracted from text strings included in the inputimage data.

In FIG. 5 that is the block diagram schematically illustrating anarrangement of the document matching process section 14, the featurescalculation section 32 calculates features (hash value and/or invariant)which are invariable relative to geometric transformation such asrotation, parallel shift, scaling up, scaling down, and the like of adocument image by use of a feature point calculated by the feature pointcalculation section 31.

FIG. 7 is a block diagram schematically illustrating an arrangement ofthe features calculation section 32. As illustrated in FIG. 7, thefeatures calculation section 32 includes a feature point extractionsection 32 a, an invariant calculation section 32 b, and a hash valuecalculation section 32 c.

As illustrated in FIG. 11, the feature point extraction section 32 aregards only one feature point as a target feature point andsequentially extracts peripheral feature points around the targetfeature point in such order that a feature point nearer to the targetfeature point is more preferentially extracted so as to extract apredetermined number of feature points (four feature points herein). InFIG. 11, four feature points h, c, d, and e are extracted as peripheralfeature points in a case where a feature point a is regarded as a targetfeature point, and four feature points a, c, e, and f are extracted asperipheral feature points in a case where a feature point b is regardedas a target feature point.

Further, the feature point extraction section 32 a extracts acombination of three points which can be selected from the fourperipheral feature points extracted in the foregoing manner. Forexample, as illustrated in FIG. 12 (a) to FIG. 12 (d), in a case wherethe feature point a in FIG. 11 is regarded as a target feature point P1,a combination of three feature points out of b, c, d, and e, that is, acombination of peripheral feature points b, c, and d, a combination ofperipheral feature points b, c, and e, a combination of peripheralfeature points b, d, and e, and a combination of peripheral featurepoints c, d, and e are extracted.

Next, the invariant calculation section 32 b calculates an invariant(one of features) Hij of the extracted combination relative togeometrical transformation.

Herein, “i” represents the number of target feature point(s) (i is aninteger not less than 1), and “j” represents the number of combinationsof three peripheral feature points (j is an integer not less than 1). Inthe present embodiment, a ratio of lengths of two lines connecting theperipheral feature points is set as the invariant Hij.

The lengths of the lines are calculated in accordance with coordinatevalues of the peripheral feature points. For example, in FIG. 12 (a),when a length of a line connecting the feature point b and the featurepoint c is A11 and a length of a line connecting the feature point b andthe feature point d is B11, the invariant H11 is such that H11=A11/B11.

In FIG. 12 (b), when a length of a line connecting the feature point band the feature point c is A12 and a length of a line connecting thefeature point b and the feature point e is B12, the invariant H12 issuch that H12=A12/B12. Moreover, in FIG. 12 (c), when a length of a lineconnecting the feature point b and the feature point d is A13 and alength of a line connecting the feature point b and the feature point eis B13, the invariant H13 is such that H13=A13/B13. Further, in FIG. 12(d), when a length of a line connecting the feature point c and thefeature point d is A14 and a length of a line connecting the featurepoint c and the feature point e is B14, the invariant H14 is such thatH14=A14/B14. In this manner, the invariants H11, H12, H13, and H14 arecalculated in the examples illustrated in FIGS. 12 (a) to (d).

In the foregoing examples, a line connecting a peripheral feature pointfirst-nearest to the target feature point and a peripheral feature pointsecond-nearest to the target feature point is indicated as Aij and aline connecting a peripheral feature point first-nearest to the targetfeature point and a peripheral feature point third-nearest to the targetfeature point is indicated as Bij, but the definition is not limited tothis, and the lines used to calculate the invariant Hij may be set inany manner.

Next, the hash value calculation section 32 c calculates a remainder ofthe following expression (2):Hi=(Hi1×10³ +Hi2×10² +Hi3×10¹ +Hi4×10⁰)/D  (2)as a hash value (one of features) Hi and stores the calculated Hi intothe memory 8. Note that, D is a constant number which is set beforehandaccording to a range which is to be set as a remainder value range.

Note that, how to calculate the invariant Hij is not particularlylimited. For example, a value calculated in accordance with a compoundratio of five points in the vicinity of the target feature point, or avalue calculated in accordance with a compound ratio of five pointsextracted from n points in the vicinity of the target feature point (nis such an integer that n≧5), or a value calculated in accordance withan arrangement of m points extracted from n points in the vicinity ofthe target feature point (m is such an integer that m<n and m≧5) and acompound ratio of five points extracted from the m points may be set asthe invariant Hij relative to the target feature point. Note that, thecompound ratio is a value calculated from four points in a straight lineor from five points on a plane and is known as an invariant relative toprojective transformation which is a kind of geometric transformation.

Further, calculation of the hash value Hi is not limited to calculatinga remainder of the expression (2) and regarding the remainder as thehash value Hi, and other hash function (for example, any one of hashfunctions mentioned in Patent Document 4) may be used.

Further, when extraction of peripheral feature points around a targetfeature point and calculation of the hash value Hi are completed, eachsection of the features calculation section 32 focuses on anotherfeature point to change the target feature point and performs extractionof peripheral feature points and calculation of a hash value. In thismanner, each section of the features calculation section 32 calculateshash values corresponding to all the feature points.

In FIG. 11, when extraction of peripheral feature points around afeature point a regarded as a target feature point and calculation of ahash value are completed, peripheral feature points around a featurepoint b regarded as a target feature point are extracted and a hashvalue is calculated. In FIG. 11, four feature points a, c, e, and f areextracted as peripheral feature points in the case where the featurepoint b is regarded as a target feature point P2.

Further, as illustrated in FIGS. 13 (a) to (d), a combination of threepoints (peripheral feature points a, e, and f, peripheral feature pointsa, e, and c, peripheral feature points a, f, and c, peripheral featurepoints e, f, and c) selected from the peripheral feature points a, c, e,and f is extracted and a hash value corresponding to the combination iscalculated, and the calculated hash value is stored in the memory 8.Further, this process is repeated so as to correspond to the number ofthe feature points, thereby calculating hash values corresponding to therespective feature points each of which is regarded as the targetfeature point. Then, the calculated hash values are stored in the memory8.

Note that, in a case of storing the input image data as a referenceimage, the features calculation section 32 transmits the hash value(features (feature vectors)) calculated in the foregoing manner andcorresponding to each feature point of the input image data to thestorage process section 35 illustrated in FIG. 5.

The storage process section 35 sequentially stores (i) hash valuescalculated by the features calculation section 32 and corresponding tothe respective feature points and (ii) indices (document IDs) eachindicative of a document (input image data) to a hash table (not shown)provided in the memory 8 in such a manner that the hash values and theindices are related to each other (see FIG. 14 (a)). In a case where thehash values have already been stored, the document IDs are stored withthem respectively corresponding to the hash values. Document IDs areserially assigned to respective documents without any duplication.

Note that, in a case where the number of documents stored in the hashtable is larger than a predetermined value (for example, 80% of thenumber of documents which can be stored), old document IDs may besearched and sequentially deleted. Further, the deleted document IDs maybe reused as document IDs of new input image data. Further, in a casewhere the calculated hash values are equal to each other (H1=H5 in FIG.14 (b)), these values may be collectively stored into the hash table.

In a case of determining whether the image of the input image data isidentical with an image of a reference image that has been alreadystored, the features calculation section 32 transmits, to the votingprocess section 33, the hash value calculated in the foregoing mannerand corresponding to each feature point of the input image data.

The voting process section 33 compares the hash value calculated fromthe input image data and corresponding to each feature point with thehash value stored in the hash table so as to vote for a reference imagehaving the same hash value (see FIG. 15). FIG. 15 is a graphillustrating an example of the number of votes for reference images ID1,ID2, and ID3. In other words, the voting process section 33 counts, withrespect to each reference image, how many times the same hash value as ahash value of the reference image is calculated from the input imagedata, and the counted number is stored in the memory 8.

Further, in FIG. 14 (b), H1 is equal to H5, and these hash values arecollectively stored into the hash table as H1. In such a table value, ina case where hash values of the input document calculated from the inputimage data includes H1, the document ID1 obtains two votes.

The similarity determination process section 34 reads out a vote resultof the voting process section 33 (an index of each reference image andthe number of votes for each reference image) from the memory 8, andextracts the maximum number of votes obtained and an index of areference image with the maximum number of votes obtained.

Further, the similarity determination process section 34 compares themaximum number of votes obtained, that indicates similarity, with apredetermined threshold value TH so as to determine whether there is anysimilarity or not (whether the input image data is identical with imagedata of the reference image or not). That is, in a case where themaximum number of votes obtained is not less than the predeterminedthreshold value TH, it is determined that “there is a similarity”, andin a case where the maximum number of votes is less than the thresholdvalue, it is determined that “there is no similarity”.

The similarity determination process section 34 then sends adetermination signal indicative of the determination result to thecontrol section 7. In the case where there is a similarity, the controlsection 7 supplies a control signal to control an output process, forexample, prohibiting an output process (an image forming process in thecolor copying machine) with respect to the image data of the inputdocument, or degrading an image quality.

Alternatively, it may be so arranged that: the similarity determinationprocess section 34 divides the number of votes obtained for eachreference image by a total number of votes (total number of featurepoints extracted from the input image data) and normalizes the result soas to calculate the similarity, thereby comparing the similarity withthe predetermined threshold value TH (80% of the number of total votesfor example) to determine the similarity.

Further, it may be so arranged that: the similarity determinationprocess section 34 divides the number of votes obtained for eachreference image by the number of times of storing a hash value (maximumnumber of times a hash value is stored) corresponding to a referenceimage whose hash value is most frequently stored and normalizes theresult so as to calculate the similarity, thereby comparing thesimilarity with the predetermined threshold value TH (80% of the numberof total votes for example) to determine the similarity.

That is, in a case where the calculated similarity is not less than athreshold value TH, it is determined that “there is a similarity”, andin a case where the calculated similarity is less than the thresholdvalue TH, it is determined that “there is no similarity”. Note that, inthis case, the total number of hash values extracted from the inputimage data may be larger than the maximum number of times of storing ahash value (particularly, a case where the document and/or the referenceimage partially has a handwritten part), so that the calculated value ofthe similarity may exceed 100%.

Further, the threshold value TH in determining the similarity may beconstant for each reference image or may be set for each reference imagein accordance with importance or the like of the reference image. As tothe importance of the reference image, for example, a paper currency, avaluable stock certificate, a top-secret document, a restricteddocument, and the like are regarded as having maximum importance, and asecret document is regarded as being less important than a papercurrency or the like. In this manner, the importance may be set bystages according to each reference image.

In this case, a weighting coefficient according to importance of areference image is stored in the memory 8 with the weighting coefficientcorresponding to an index of the reference image, and the similaritydetermination process section 34 determines the similarity by using thethreshold value TH corresponding to the reference image with the maximumnumber of votes obtained.

Further, it may be so arranged that: in determining the similarity, thethreshold value TH is made constant and the number of votes for eachreference image (the number of votes obtained for each reference image)is multiplied by a weighting coefficient of each reference image so asto determine the similarity.

In this case, the weighting coefficient according to the importance ofeach reference image is stored in the memory 8 with the weightingcoefficient corresponding to an index of each reference image, and thesimilarity determination process section 34 calculates a correctednumber of obtained votes by multiplying the number of obtained votes ofeach reference image by the weighting coefficient of the referenceimage, thereby determining the similarity in accordance with thecorrected number of obtained votes.

For example, a maximum corrected number of obtained votes may becompared with the threshold value TH, or a value obtained by normalizingthe maximum corrected number of obtained votes by the number of totalvotes may be compared with the threshold value TH, or a value obtainedby normalizing the maximum corrected number of obtained votes by themaximum number of times of storage may be compared with the thresholdvalue TH. Further, in this case, for example, the weighting coefficientis set to be more than 1 and to be larger as the importance of thereference image is higher.

Further, in the present embodiment, a single hash value is calculatedfor a single feature point (target feature point), but the presentinvention is not limited to this, and it may be so arranged that aplurality of hash values are calculated for a single feature point(target feature point). For example, it may be so arranged that: sixpoints are extracted as peripheral feature points around the targetfeature point, and three points are extracted from five points for eachof six combinations obtained by extracting five points from the sixpoints, so as to calculate an invariant, thereby calculating a hashvalue. In this case, six hash values are calculated for a single featurepoint.

Next, explained is the centroid calculation section 45 provided in thefeature point calculation section 31 in detail. In a conventional imagematching apparatus, a threshold value as a criterion for determiningwhether or not a connected region is the one to be subjected to featurescalculation is fixed to a default threshold value. The default thresholdvalue is a value that is capable of removing an isolated dot and a noisein image data of a reference document.

From this reason, in a case where an image of an N-up document, areduced-size document, or the like is reduced in size from its originalimage, when a connected region is extracted by use of the same thresholdvalue as the reference document, in a part in which the number of pixelsactually exceeds the threshold value and which should be extracted as aconnected region, the number of pixels cannot exceed the threshold valuebecause the image is reduced in size, thereby resulting in that the partis not extracted as the connected region. Consequently, the number offeature points decreases and features differ, thereby decreasingaccuracy in determination.

In view of this, in the present embodiment, as has been alreadydescribed, in the case where a target document to be matched is not anN-up document, the centroid calculation section 45 determines, by use ofthe default threshold value, whether or not a connected region is theone to be subjected to the centroid calculation, in a similar manner tothe conventional apparatus. On the other hand, in the case where thetarget document is an N-up document, the centroid calculation section 45determines whether or not a connected region is the one to be subjectedto the centroid calculation, by use of a variant threshold which variesdepending on the number of images laid out on the N-up document and adocument size of input image data.

FIG. 1 is a block diagram schematically illustrating an arrangement ofthe centroid calculation section 45. The centroid calculation section 45includes: a labeling process section (a connected region specifyingsection) 45 a, a threshold value processing section 45 b, a centroidcalculation process section (a feature point calculation processsection) 45 c. Note that processes in each section can be carried out byuse of a method disclosed in Japanese Patent Application, Tokugan, No.2006-126761.

The labeling process section 45 a carries out labeling with respect to atarget pixel to be labeled by referring to labels assigned to adjacentpixels. A labeling process in the labeling process section 45 a isdescribed below with reference to FIGS. 16 through 19.

As illustrated in FIG. 16 (a), a label buffer having two lines holdslabels of pixels in one-previous line and labels of pixels in a currentline. Then, as illustrated in FIG. 17, labeling is carried out withrespect to a current pixel X by referring labels of four adjacent pixelsthat are adjacent to the current pixel X and placed in an L shape.

When all of the four pixels adjacent to the current pixel X are notlabeled, a label that is not used is assigned to the current pixel X. Onthe other hand, when the four pixels adjacent to the current pixel Xhave the same label, the same label as the four pixels is assigned tothe current pixel X.

Further, when the four pixels adjacent to the current pixel X havedifferent labels, the oldest label among the labels is assigned to thecurrent pixel X (merge). In this case, these labels are recorded in alabel equivalent table as the same value (see FIG. 18).

According to the process, in the case of the example of FIG. 16 (a), alabel of the current pixel X is “2”.

While carrying out labeling, the labeling process section 45 a addscoordinate values of target pixels to be labeled to a buffer foraccumulating coordinate values according to each label, as illustratedin FIG. 16 (b).

Explained is a nine-pixel region D in which a label “1” is assigned toeach pixel, illustrated in FIG. 16 (a), as an example.

Accumulated values of X-coordinate and Y-coordinate in the nine-pixelregion D are as follows:

X-coordinate: (3+4)+(3+4)+(4+5)+(4+5+6)=38

Y-coordinate: (3+3)+(4+4)+(5+5)+(6+6+6)=42.

Accordingly, a centroid is such that:

X-coordinate: 38/9 nearly equal to 4.22

Y-coordinate: 42/9 nearly equal to 4.67.

With respect to the label “2”, FIG. 16 (b) shows a result of threepixels that have been already labeled.

In a case where the adjacent pixels have different labels, the labelingprocess section 45 a carries out a merging process in which coordinatevalues that are accumulated respectively in two labels are added to thebuffer so that two labels are merged.

Further, each label has a flag, and when the process is completed withrespect to one line, the labeling process section 45 a checks whethercoordinate values have been added to the buffer or not. As a state ofthe flag, there are three states: “empty”, “processed”, and “notprocessed”. The “empty” is a state where a label is not used. When alabel is assigned to a current pixel, a flag of the label is changed tothe “processed”. Further, in a case where the label “1” has been usedbut not used in a current line at a time when the process is completedwith respect to the current line, the “processed” state is changed tothe “not processed” state. Every time when the process starts withrespect to a line, the labels are initialized (a label that is not usedis initialized as the “empty”, and a label that has been used isinitialized as “processed”).

The threshold value processing section 45 b (i) compares the number ofpixels included in connected regions specified by the labeling processsection 45 a with a threshold value, (ii) extracts a connected region inwhich the number of the pixels exceeds the threshold value, from theconnected regions thus specified, and (iii) sends the connected regionthus extracted to the centroid calculation process section 45 c. Thecentroid calculation process section 45 c calculates a centroid of theconnected region received from the threshold value processing section 45b, as described later.

In the present embodiment, in a case where a target document to bematched is determined as an N-up document by the document typediscrimination section 13, the threshold value processing section 45 buses, instead of the default threshold value, a variant threshold valuewhich varies depending on the number of images laid out on the targetdocument and its document size.

Here, the number of images laid out on an N-up document is detected bythe document type discrimination section 13, and a document size isdetected by the document size detection section 40, as has been alreadydescribed. The document size detection section 40 may adopt, forexample, a method of detecting, in main and sub scanning directions, asize of a document placed on a scanner platen by use of aphotoelectronic conversion element such as a phototransistor provided inthe color image input apparatus 2, or a method in which a controlsection detects a document size selected by a user from the operationpanel 6.

FIG. 19 shows one example of the variant threshold value. Here, anA4-size reference image data is set to a resolution of 300 dpi. Thevariant threshold value is expressed as a function of the defaultthreshold value, and a multiplier coefficient Z is set so as to varydepending on a document size of an N-up document and the number ofimages laid out thereon.

The multiplier coefficient Z is a value corresponding to a reductionratio from an original image size. For example, in a case where adocument size of input image data is A4 and the number of images laidout on the input image data is 2 (2-in-1 document), the two documentimages laid out on the input image data are respectively reduced about0.7 times from their original image sizes. Accordingly, the multipliercoefficient Z is set to 0.7, and the variant threshold value is suchthat the default threshold value×0.7.

With the arrangement, the criterion for determining whether or not thecentroid calculation is carried out with respect to a specifiedconnected region is reduced in conformity to a reduction ratio of thetarget document, so that the threshold value is set so as to variesdepending on the reduction ratio of the target document. This solvestrouble in which a feature point is not easily extracted in an N-updocument, thereby improving accuracy in matching.

Further, in the example of FIG. 19, in a case where a document size ofinput image data is A3 and the number of images laid out on the inputimage data is 2 (2-in-1 document), each size of the two document imageslaid out on the input image data is equal to A4 size, i.e., theiroriginal document size. Accordingly, in this case, the default thresholdvalue can be used, that is, the multiplier coefficient Z is 1.

Furthermore, for example, in a case where a document size of input imagedata is A3 and the number of images laid out on the input image data is8 (8-in-1 document), each size of the eight document images laid out onthe input image data is reduced about 50% from A4 size, i.e., theiroriginal document size. Accordingly, the multiplier coefficient Z is setto 0.5, and the variant threshold value is such that the defaultthreshold value×0.5.

In the present embodiment, even in a case where a target document to bematched is not an N-up document, when a document size of the targetdocument is smaller than a document size of a reference documentaccording to a detection result of the document size detection section40 (i.e., a case where a document image is considered to be reduced),the threshold value processing section 45 b may use, instead of thedefault threshold value, the variant threshold value which variesdepending on its reduction ratio.

For example, in a case where an A4-size reference document is set to aresolution of 300 dpi, when a document size of input image data is A5,it is assumed that its document image is reduced 70%, similarly to a2-in-1 document. In this case, the variant threshold value that is setsuch that the default threshold value×0.7 may be used.

The centroid calculation process section 45 c calculates a centroid bydividing accumulated coordinate values by the number of pixels. Inregard to a label for which no coordinate values are added to the bufferat the time when the process with respect to one line has beencompleted, a connected component of the label does not further continuein the subsequent lines. From this reason, the centroid calculationprocess section 45 c calculates a centroid of the connected region bydividing accumulated coordinate values by the number of pixels. Afterthe calculation, the label is initialized.

In calculating a centroid, the centroid calculation process section 45 ccalculates a centroid with respect to a connected region in which thenumber of pixels exceeds the default threshold value or the variantthreshold value, in accordance with the threshold value processingresult by the threshold value processing section 45 b.

The following explains a similarity determination process in the digitalcolor copying machine 1 with reference to the flowchart in FIG. 20.

Upon receiving an instruction input entered by a user from the operationpanel 6, the control section 7 determines whether a storage mode isselected or not (S1). In a case where the storage mode is selected,input image data is stored.

Firstly, the control section 7 controls each section of the documentmatching process section 14 so as to leave the threshold value as adefault threshold value, which is a criterion for determining whether ornot a centroid of a connected region is calculated (S2), and cause athreshold value process at the default threshold value for calculating afeature point (S3), sequentially, a features calculation process (S4),and a document ID storage process (S5).

On the other hand, in a case where it is determine that the storage modeis not selected, that is, a matching mode is selected in S1, it isdetermined whether a target document to be matched is an N-up documentor not (S6). When it is determined that the target document is not anN-up document in S6, the control section 7 controls each section of thedocument matching process section 14 so as to leave the threshold valueas the default threshold value, which is the criterion for determiningwhether or not a centroid of a connected region is calculated, similarlyto the storage mode (S13), and cause a threshold value process at thedefault threshold value for calculating a feature point (S8). When it isdetermined that the target document is an N-up document in S6, thecontrol section 7 controls each section of the document matching processsection 14 so as to change the threshold value, which is the criterionfor determining whether or not a centroid of a connected region iscalculated, to a variant threshold value that varies depending on adocument size of the target document and the number of images laid outon the N-up document (S7), and cause a threshold value process forcalculating a feature point, by use of the variant threshold value thatvaries depending on the document size and the number of the images laidout on the N-up document (S8).

Then, on the basis of the feature point calculated in S8, a featurescalculation process (S9), a voting process (S10), and a similaritydetermination process (S11) are carried out. When it is determined thatthere is a similarity in S11, the control section 7 supplies a controlsignal so that an output process is carried out with respect to theinput image data in accordance with a control on the reference documentsimilar to the target document (the input image data) (S12).

As described above, in the digital color copying machine 1 of thepresent embodiment, in the case where the input image data is an N-updocument in the matching mode, the centroid calculation section 45 inthe feature point calculation section 31 carries out the threshold valueprocess by changing a threshold value as the criterion for determiningwhether or not a centroid of a connected region is calculated to thevariant threshold value that varies depending on a document size of theN-up document and the number of images laid out thereon, in other words,a variant threshold value which varies depending on a reduction ratio ofeach document image, and which can be calculated from the document sizeand the number of the images laid out on the N-up document.

Moreover, when it is determined that the input image data is the one ofa reduced-size document in the matching mode, the centroid calculationsection 45 in the feature point calculation section 31 carries out thethreshold value process by changing a threshold value as the criterionfor determining whether or not a centroid of a connected region iscalculated to the variant threshold value which varies depending on areduction ratio of the reduced-size document, and which can becalculated from a document size of the target document and a documentsize of a reference document.

With the arrangement, even when a target document to be matched is anN-up document and images laid out thereon are reduced in size, or thetarget document is a reduced-size document, a threshold value as thecriterion for determining whether or not a centroid of a connectedregion is calculated is set to a threshold value which varies dependingon a reduction ratio of the target document. This can improve accuracyin matching in an N-up document.

Further, although this embodiment deals with a case where the presentinvention is applied to the digital color copying machine 1, the targetapparatus to which the present invention is applied is not limited tothe above apparatus, and may be, for example, a digital colormulti-function printer (MFP) 100, as illustrated in FIG. 21. The digitalcolor multi-function printer 100 has functions such as a copyingfunction, a printer function, a facsimile function, a scanner function,and a scan to e-mail function.

In FIG. 21, members having the same functions as those in the digitalcolor copying machine 1 have the same referential numerals, and are notdescribed here.

Here, a communication device 5 is constituted by, for example, a modemand a network card. The communication device 5 performs a datacommunication with other devices connected to a network (e.g., apersonal computer, a server, other digital multi-function printer, afacsimile device) via a network card, a LAN cable, or the like.

In transmitting image data, the communication device 5 (i) ensures atransmittable state by carrying out a transmission procedure with adestination device, (ii) reads out image data compressed in apredetermined format (image data scanned by a scanner) from a memory,(iii) carries out necessary processes such as a conversion of acompression format with respect to the image data, and (iv) sequentiallytransmits the image data to the destination device via a communicationline.

Further, in receiving image data, while carrying out a communicationprocedure, the communication device 5 receives image data transmittedfrom an originating communication device, and sends the image data to acolor image processing apparatus 3. The image data thus received by thecolor image processing apparatus 3 is subjected to predeterminedprocesses such as a decompression process, a rotating process, aresolution conversion process, an output tone correction, and a tonereproduction process. The image data thus processed is outputted by acolor image output apparatus 4. Further, it may be so arranged that thereceived image data is stored in a storage device (not shown), and thecolor image processing apparatus 3 reads out the image data as requiredand carries out the predetermined processes with respect to the imagedata.

In the multi-function printer 100, a user can input, from an operationpanel 6, a processing request (e.g., a processing mode (copy, printing,transmission, editing and the like), the number of images to beprocessed (the number of images to be copied, the number of images to beprinted out), a destination of input image data, and the like) withrespect to the image data. In a case where it is determined that thereis a similarity, a control section 7 controls a document matchingprocess section 14 in regard to not only a copying process, but alsooutput processes such as printing, transmission, editing and the like.

For example, in a case where a facsimile transmission mode is selected,when the document matching process section 14 determines prohibition ofan output of input image data, the image data stored in a memory isdeleted and a facsimile transmission of the image data is not carriedout. Alternatively, in a case where, even though the input image data isidentical with a reference document, a facsimile transmission of theimage data is permitted (a document ID and a destination of the imagedata are stored in advance so as to be related to each other), the imagedata may be transmitted by referring to information of the destination.

A facsimile process in the digital color multi-function printer 100 isexplained with reference to FIG. 22. In FIG. 22, process sections thatdo not carry out a process are illustrated in dot lines. In thisembodiment, a segmentation process section 17 is used in the facsimileprocess. However, the segmentation process section 17 is not necessarilyrequired in the facsimile process. Further, in facsimile transmission, aresolution conversation section and a compression/decompression processsection carry out each process after a tone reproduction section 22.

In transmission of input image data, for example, 8-bit input image dataloaded by a color image input apparatus 2 is subjected to each of theaforementioned processes in a color image processing apparatus 3, andRGB signals of the input image data thus processed are converted into aluminance signal (a K signal in FIG. 22), for example, by a matrixcalculation or the like, in an input tone correction section 15. Theimage data thus converted into the format of the luminance signal isthen subjected to predetermined processes in a segmentation processsection 17 and a spatial filter process section 20, and binarized, forexample, by an error diffusion process in a tone reproduction section22. The image data thus binarized is then subjected to a rotatingprocess as needed, compressed into a predetermined format in thecompression/decompression process section, and stored in a memory (notshown).

When a communication device (e.g., a modem) 5 carries out a transmissionprocedure with a destination device so as to ensure a transmittablestate, the image data compressed in the predetermined format is read outfrom the memory, subjected to necessary processes such as a conversionof the compression format, and sequentially transmitted to thedestination device via a communication line.

On the other hand, in reception of image data, when an image istransmitted from an originating communication device via a communicationline, while carrying out the communication procedure, a control sectionreceives the image transmitted from the originating communicationdevice. Thus received image data compressed in a predetermined format issent to the color image processing apparatus 3.

The compression/decompression process section decompresses the imagedata thus received in the color image processing apparatus 3 so that thedocument image received as a page image is reproduced. Thus reproduceddocument image is then subjected to a rotating process depending on acapability of a color image output apparatus and is then subjected to aresolution conversion process in a resolution conversion section.Finally, the color image output apparatus outputs the image datareproduced as an image per page.

The above explanation deals with a case of processing monochrome imagedata as an example, but the present invention is not limited to this.For example, image data may be also processed by use of a color imageprocessing apparatus that includes, between the segmentation processsection 17 and the spatial filter process section 20, (i) a colorcorrection section 18 that carries out, with respect to RGB signalsloaded by the color image input apparatus 2, a process of removing colorimpurity on the basis of spectral characteristics of CMY (C: Cyan, M:Magenta, and Y: Yellow) color materials of toner or ink that include anuseless absorption component, in order to realize a faithful colorreproduction, and (ii) a black generation and under color removalsection that carries out (a) a black generation process for generating ablack (K) signal from three color (CMY) signals after the colorcorrection process and (b) a generation process of new CMY signals byremoving the K signal obtained by the black generation process from theoriginal CMY signals.

Further, in the case of the digital color copying machine 1, input imagedata is the one scanned and inputted from a document by a scanner.However, in the digital color multi-function printer 100, the inputimage data encompasses image data that is scanned and inputted from adocument by a scanner, and electronic data formed by use of a computer(software), e.g., electronic data formed by filling in a form withnecessary information by use of a computer (software). In view ofpractical use, there are two types of the electronic data: (i)paper-based data that is scanned and filed, and (ii) data that is formedas electronic data directly (an electronic application and the like).

Moreover, as to electronic data of an N-up document, a setting of anN-up document may be carried out by application software, oralternatively N-up printing may be set on a setting screen of a printerdriver. In either case, image data is converted into PDL (pagedescription language) and developed into bitmap data at a time when theimage data is received in the digital color multi-function printer 100.Accordingly, it is possible to determine whether the bitmap data is anN-up document or not in the similar manner to the similaritydetermination with respect to image data inputted by a scanner.

In the arrangement of FIG. 21, a similarity determination process iscarried out in a document matching process section 14 provided in thedigital color multi-function printer 100. However, the present inventionis not limited to this. For example, a part of or all functions that thecontrol section 7 and the document matching process section 14 have maybe carried out in an external apparatus, which is connected to thedigital color multi-function printer 100 so that the external apparatusbe communicated with the digital color multi-function printer 100.

Furthermore, the present invention may be applied, for example, to amonochrome multi-function machine. Further, the present invention may beapplied not only to multi-function machines, but also to a facsimilecommunication apparatus, a copying machine, an image readout apparatus,and the like.

For example, FIG. 23 is a block diagram illustrating an exemplaryarrangement of a flat bed scanner 101 to which the present invention isapplied.

As illustrated in FIG. 23, a flat bed scanner 101 includes a color imageinput apparatus 2, and a color image processing apparatus 3′. The colorimage processing apparatus 3′ includes an A/D conversion section 11, ashading correction section 12, a document matching process section 14, acontrol section (not shown in FIG. 23), and a memory (not shown in FIG.23), and the color image input apparatus 2 is connected thereto. As suchan image data output processing apparatus is formed. Note that, the A/Dconversion section 11, the shading correction section 12, the documentmatching process section 14, the control section 7, and the memory 8 inthe color image input apparatus (image readout means) 2 respectivelyhave almost the same functions as those in the digital color copyingmachine 1, and are not described here in detail.

Each section (each block) constituting the document matching processsection and the control section included in the digital color copyingmachine 1, the multi-function printer 100, or the flat bed scanner 101may be realized by software with the use of a processor such as a CPU.Namely, the digital color copying machine 1, the digital colormulti-function printer 100, or the flat bed scanner 101 includes: a CPU(central processing unit) for executing a program for realizing eachfunction; a ROM (read only memory) that stores the program; a RAM(random access memory) that develops the program; a storage device(storage medium) such as a memory in which the program and various dataare stored; and the like. The object of the present invention can berealized in such a manner that the digital color copying machine 1, thedigital color multi-function printer 100, or the flat bed scanner 101 isprovided with a computer-readable storage medium for storing programcodes (such as executable program, intermediate code program, and sourceprogram) of programs of the digital color copying machine 1, the digitalcolor multi-function printer 100, or the flat bed scanner 101 includeswhich programs serve as software for realizing the functions, and acomputer (alternatively, CPU or MPU) reads out and executes the programcodes stored in the storage medium.

The storage medium is, for example, tapes such as a magnetic tape and acassette tape, or discs such as magnetic discs (e.g. a floppy Disc® anda hard disc), and optical discs (e.g. CD-ROM, MO, MD, DVD, and CD-R).Further, the storage medium may be cards such as an IC card (including amemory card) and an optical card, or semiconductor memories such as maskROM, EPROM, EEPROM, and flash ROM.

Further, the digital color copying machine 1, the multi-function printer100, or the flat bed scanner 101 may be arranged so as to be connectableto a communication network so that the program code is supplied to thedigital color copying machine 1, the multi-function printer 100, or theflat bed scanner 101 through the communication network. Thecommunication network is not particularly limited. Examples of thecommunication network include the Internet, intranet, extranet, LAN,ISDN, VAN, CATV communication network, virtual private network,telephone network, mobile communication network, and satellitecommunication network. Further, a transmission medium that constitutesthe communication network is not particularly limited. Examples of thetransmission medium include (i) wired lines such as IEEE 1394, USB,power-line carrier, cable TV lines, telephone lines, and ADSL lines and(ii) wireless connections such as IrDA and remote control using infraredray, Bluetooth®, 802.11, HDR, mobile phone network, satelliteconnections, and terrestrial digital network. Note that the presentinvention can be also realized by the program codes in the form of acomputer data signal embedded in a carrier wave, which is the programthat is electrically transmitted.

Furthermore, each block of the digital color copying machine 1, themulti-function printer 100, or the flat bed scanner 101 is not limitedto the one that is realized by use of software, and may be realized byhardware logic. Each block of the digital color copying machine 1, themulti-function printer 100, or the flat bed scanner 101 may be acombination of hardware carrying out some of the processes and thecomputing means controlling the hardware and executing program code forthe other processes.

The computer system of the present invention may include: an image inputapparatus such as a flat bed scanner, a film scanner, and a digitalcamera; a computer loaded with a predetermined program to executeprocesses such as the similarity calculation process and the similaritydetermination process; an image display apparatus, such as a CRT displayand a liquid crystal display, for displaying a result of the process bythe computer; and an image forming apparatus, such as a printer, foroutputting a result of the process by the computer on a paper etc.Furthermore, a network card or a modem may be provided as communicationmeans to be connected with a server etc. via a network.

As described above, a first image matching apparatus of the presentinvention includes: a features extraction section for extractingfeatures of an image from input image data, the features extractionsection extracting the features based on a connected region in whichpixels in a binarized image of the image data are connected to eachother and the number of the pixels exceeds a default threshold value,the apparatus (i) causing the features extraction section to extract thefeatures from input image data of a target document to be matched, and(ii) determining whether or not the target document has a similar imageto that of a preliminarily stored reference document based on acomparison of the features thus extracted with features of the image ofthe reference document. The apparatus, further includes: an N-updocument determination section that determines (i) whether or not thetarget document is an N-up document on which plural documents are laidout, and (ii) the number of images laid out on the N-up document when itis determined that the target document is an N-up document; and adocument size detection section that detects a document size of thetarget document, when the N-up document determination section determinesthat the target document is an N-up document, the features extractionsection extracting the features, by use of not the default thresholdvalue but a variant threshold value that varies depending on the numberof the images laid out on the N-up document and the document size thatare found and detected by the N-up document determination section andthe document size detection section, respectively, based on a connectedregion in which the number of pixels exceeds the variant thresholdvalue.

Image data that is inputted into the image matching apparatus includes:image data that is scanned and inputted by a scanner; and further,electronic data formed by use of a computer (software), e.g., electronicdata that is formed by filling in a form of the electronic data withnecessary information by use of a computer (software).

With the arrangement, even when the target document is an N-up documentand images thereof are reduced in size, a threshold value that is usedfor extracting a connected region to be subjected to featurescalculation is the variant threshold value that varies depending on thedocument size and the number of the images laid out on the N-updocument, that is, a threshold value that varies depending on areduction ratio of the target document, thereby improving accuracy inmatching in an N-up document.

In the first image matching apparatus of the present invention, thefeatures extraction section can include: a connected region specifyingsection that specifies a connected region in which pixels in a binarizedimage are connected to each other; a threshold value processing sectionthat extracts a connected region in which the number of pixels exceeds athreshold value, based on a comparison of the number of pixels includedin the connected region thus specified by the connected regionspecifying section with the threshold value; a feature point calculationprocess section that calculates a feature point of the connected regionextracted by the threshold value processing section; and a featurescalculation section that calculates features of an image by use of thefeature point calculated by the feature point calculation processsection, when it is determined that the target document is an N-updocument, the threshold value processing section using, as the thresholdvalue, the variant threshold value which varies depending on the numberof the images laid out on the target document and the document size,instead of the default threshold value.

A first image data output processing apparatus of the present inventionthat carries out an output process with respect to image data, includes:the first image matching apparatus of the present invention; and anoutput process control section that controls an output process withrespect to image data of the target document in accordance with adetermination result of the image matching apparatus.

As described above, the first image matching apparatus of the presentinvention can determine a similarity with high accuracy, even in a casewhere the target document is an N-up document and images thereof arereduced in size from their original size.

Accordingly, in a case where an image of image data to be subjected tothe output process is similar to that of a reference document, the imagedata output processing apparatus including such the image matchingapparatus can control an output process with high accuracy. This canincrease reliability.

An image matching method of the present invention includes the step of:(a) extracting features of an image from input image data based on aconnected region in which pixels in a binarized image of the image dataare connected to each other and the number of the pixels exceeds adefault threshold value, wherein the features are extracted from inputimage data of a target document to be matched in the step (a), and it isdetermined whether or not the target document has a similar image tothat of a preliminarily stored reference document, based on a comparisonof the features thus extracted with features of the image of thereference document. The method further includes the steps of: (b)determining (i) whether or not the target document is an N-up documenton which plural documents are laid out, and (ii) the number of imageslaid out on the N-up document when it is determined that the targetdocument is an N-up document; and (c) detecting a document size of thetarget document, when it is determined that the target document is anN-up document in the step (b), the step (a) is carried out such that thefeatures are extracted, by use of not the default threshold value but avariant threshold value that varies depending on the number of theimages laid out on the N-up document and the document size that arefound and detected by the N-up document determination section and thedocument size detection section, respectively, based on a connectedregion in which the number of pixels exceeds the variant thresholdvalue.

In the similar manner to the first image matching apparatus that hasbeen already described, in the image matching method, it is possible toextract features under the same condition as the reference document evenwhen the target document is an N-up document and images are reduced insize from their original sizes. This makes it possible to accuratelydetermine a similarity to a reference document.

A second image matching apparatus of the present invention includes: afeatures extraction section for extracting features of an image frominput image data, the features extraction section extracting thefeatures based on a connected region in which pixels in a binarizedimage of the image data are connected to each other and the number ofthe pixels exceeds a default threshold value, the apparatus (i) causingthe features extraction section to extract the features from input imagedata of a target document to be matched, and (ii) determining whether ornot the target document has a similar image to that of a preliminarilystored reference document based on a comparison of the features thusextracted with features of the image of the reference document. Theapparatus, further includes: a document size detection section thatdetects a document size of the target document, when the document sizethat is detected by the document size detection section is smaller thana document size of the reference document, the features extractionsection extracting the features, by use of not the default number ofpixels but the predetermined number of pixels that varies depending onthe document size that is detected by the document size detectionsection, based on a connected region in which the number of pixelsexceeds the predetermined number of pixels that varies depending on thedocument size.

In the arrangement, even in a case where a document size of the targetdocument is smaller than a document size of the reference document andan image thereof is reduced in size, in the features extraction section,a threshold value for extracting a connected region to be subjected tofeatures calculation is set so as to vary depending on a reduction ratioof the target document. This can improve accuracy in matching in an N-updocument.

Note that, in the arrangement, the threshold value for extracting aconnected region to be subjected to features calculation may be smallerthan that of a reference document, thereby resulting in that a noisecomponent such as an isolated dot that is removed in image data of thereference document cannot be removed in a target document. However, thishas a less effect on determination, compared with a case where connectedregions to be used for extracting features are reduced, and can increaseaccuracy in determination.

A second image data output processing apparatus of the present inventionthat carries out an output process with respect to image data, includes:a second image matching apparatus of the present invention; and anoutput process control section that controls the output process withrespect to the image data of the target document in accordance with adetermination result of the image matching apparatus.

As described above, the second image matching apparatus of the presentinvention can determine a similarity with high accuracy, even in a casewhere a target image to be matched in the target document is reduced insize compared with an image of a reference document.

Accordingly, in a case where image data to be subjected to the outputprocess is similar to that of the reference document, the image dataoutput processing apparatus including such the image matching apparatuscan control the output process with high accuracy. This can increasereliability.

Furthermore, each of the image matching apparatuses may be realized by acomputer. In this case, the present invention includes an imageprocessing program that causes a computer to function as respectivesections in each of the image matching apparatuses so that each of theimage matching apparatuses is realized in the computer, and acomputer-readable storage medium in which the program is stored.

The embodiments and concrete examples of implementation discussed in theforegoing detailed explanation serve solely to illustrate the technicaldetails of the present invention, which should not be narrowlyinterpreted within the limits of such embodiments and concrete examples,but rather may be applied in many variations within the spirit of thepresent invention, provided such variations do not exceed the scope ofthe patent claims set forth below.

1. An image matching apparatus, comprising a features extraction sectionfor extracting features of an image from input image data, the featuresextraction section extracting the features based on a connected regionin which pixels in a binarized image of the image data are connected toeach other and the number of the pixels exceeds a default thresholdvalue, said apparatus (i) causing the features extraction section toextract the features from input image data of a target document to bematched, and (ii) determining whether or not the target document has asimilar image to that of a preliminarily stored reference document basedon a comparison of the features thus extracted with features of theimage of the reference document; said apparatus, further comprising: anN-up document determination section that determines (i) whether or notthe target document is an N-up document on which plural documents arelaid out, and (ii) the number of images laid out on the N-up documentwhen it is determined that the target document is an N-up document; anda document size detection section that detects a document, size of thetarget document, when the N-up document determination section determinesthat the target document is an N-up document, the features extractionsection extracting the features, by use of not the default thresholdvalue but a variant threshold value that varies depending on the numberof the images laid out on the N-up document and the document size thatare found and detected by the N-up document determination section andthe document size detection section, respectively, based on a connectedregion in which the number of pixels exceeds the variant thresholdvalue.
 2. The image matching apparatus as set forth in claim 1, wherein:the features extraction section includes: a connected region specifyingsection that specifies a connected region in which pixels in a binarizedimage are connected to each other: a threshold value processing sectionthat extracts a connected region in which the number of pixels exceeds athreshold value, based on a comparison of the number of pixels includedin the connected region thus specified by the connected regionspecifying section with the threshold value; a feature point calculationprocess section that calculates a feature point of the connected regionextracted by the threshold value processing section; and a featurescalculation section that calculates features of an image by use of thefeature point calculated by the feature point calculation processsection, when it is determined that the target document is an N-updocument, the threshold value processing section using, as the thresholdvalue, the variant threshold value which varies depending on the numberof the images laid out on the target document and the document sizeinstead of the default threshold value.
 3. A non-transitorycomputer-readable storage medium storing a computer program, whichcauses an image matching apparatus as set forth in claim 1 to operate.4. An image data output processing apparatus that carries out an outputprocess with respect to image data, the apparatus comprising: an imagematching apparatus; and an output process control section that controlsan output process with respect to image data of a target document to bematched in accordance with a determination result of the image matchingapparatus, said image matching apparatus, including: a featuresextraction section for extracting features of an image from input imagedata, the features extraction section extracting the features based on aconnected region in which pixels in a binarized image of the image dataare connected to each other and the number of the pixels exceeds adefault threshold value, said image matching apparatus (i) causing thefeatures extraction section to extract the features from input imagedata of a target document to be matched, and (ii) determining whether ornot the target document has a similar image to that of a preliminarilystored reference document based on a comparison of the features thusextracted with features of the image of the reference document: saidimage matching apparatus, further including: an N-up documentdetermination section that determines (i) whether or not the targetdocument is an N-up document on which plural documents are laid out, and(ii) the number of images laid out on the N-up document when it isdetermined that the target document is an N-up document; and a documentsize detection section that detects a document size of the targetdocument, when the N-up document determination section determines thatthe target document is an N-up document, the features extraction sectionextracting the features, by use of not the default threshold value but avariant threshold value that varies depending on the number of theimages laid out on the N-up document and the document size that arefound and detected by the N-up document determination section and thedocument size detection section, respectively, based on a connectedregion in which the number of pixels exceeds the variant thresholdvalue.
 5. An image matching method comprising the step of: (a)extracting features of an image from input image data based on aconnected region in which pixels in a binarized image of the image dataare connected to each other and the number of the pixels exceeds adefault threshold value, wherein the features are extracted from inputimage data of a target document to be matched in the step (a), and it isdetermined whether or not the target document has a similar image tothat of a preliminarily stored reference document, based on a comparisonof the features thus extracted with features of the image of thereference document; said method, further comprising the steps of: (b)determining (i) whether or not the target document is an N-up documenton which plural documents are laid out, and (ii) the number of imageslaid out on the N-up document when it is determined that the targetdocument is an N-up document: and (c) detecting a document size of thetarget document, when it is determined that the target document is anN-up document in the step (b), the step (a) is carried out such that thefeatures are extracted, by use of not the default threshold value but avariant threshold value that varies depending on the number of theimages laid out on the N-up document and the document size that arefound and detected by the N-up document determination section and thedocument size detection section, respectively, based on a connectedregion in which the number of pixels exceeds the variant thresholdvalue.